Model-agnostic feature importance and effects with dependent features: a conditional subgroup approach

Abstract The interpretation of feature importance in machine learning models is challenging when features are dependent. Permutation (PFI) ignores such dependencies, which can cause misleading interpretations due to extrapolation. A possible remedy more advanced conditional PFI approaches that enable the assessment on all other features. Due this shift perspective and order correct interpretations, it beneficial if conditioning transparent comprehensible. In paper, we propose a new sampling mechanism for distribution based permutations subgroups. As these subgroups constructed using tree-based methods as transformation trees, becomes inherently interpretable. This not only provides simple effective estimator PFI, but also local estimates within addition, apply approach partial dependence plots, popular method describing effects suffer from extrapolation dependent interactions present model. simulations real-world application, demonstrate advantages subgroup over existing methods: It allows compute true data than proposals enables fine-grained